Interactive Digital Storytelling: Towards a Hybrid Conceptual Approach

1 Introduction In this contribution, Interactive Digital Storytelling is viewed as a hybrid form of game design and cinematic storytelling for the understanding and making of future learning and entertainment applications. The paper shall present formal design models that provide a conceptual bridge between both traditional linear narrative techniques as well as agent-based emergent conversations with virtual characters. In summary, a theoretical classification of thinking models for authors and interactive experiences for users will be presented. The conceptual work is based upon practical experiments within several research projects on edutainment, which employ conversations with virtual characters to convey information and to entertain. By building several prototypes, two different approaches where explored to combine plot-based interactive storytelling with character-based emergent conversations. Visual impressions of the examples are shown in Fig. 1 and will be explained in more detail in the full paper. In both examples, several virtual animated characters converse digitally with each other and with a user who mainly types text with the keyboard, optionally complemented by choice functionality and special hardware interfaces. The resulting conversations differ in their direction of approaching a middle ground between predefined narrative presentations and emergent conversations with a user, by combining emerging chatterbot dialogues with a story structure. The user experiences a semi-autonomous behaviour of interacting agents. This paper is not about the difference between stories and games. The motivation is on the potential of both to offer structures for learning and entertainment. Instead of trying to draw a distinct line between them, conceptual models for authors have to be defined, who are responsible to flesh out a suitable design within a variability of forms. Design elements include aspects of drama and filmmaking, dialogue design, as well as game design and game tuning. The actual challenge for the design of learning applications with conversational agents is the necessity that authors have to take on responsibility concerning the intended outcome and effect. In fact, they have to balance the bias between a pre-structured storyline (and possibly a timeline) which they may have strictly defined, and the agency that users shall experience through the design of the author. However, there is no one-dimensional borderline between both. In the following, the paper presents a model with several levels which shall help to form a more differentiated picture. 2 Conceptual Models for Storytelling and Agency In Fig. 2, a traditional modus operandi is sketched at four abstract levels. The distinction between levels may vary from project to project. The four levels were found to be suitable for the addition of interaction at each of the levels to form a classification of genres. On the top level of highest story abstraction, the overall dramatic outline is sketched. For example, there may be a hero’s journey in 3 acts, or a Propp model. Further, authors break down the story into scenes which are handled at the next level. Each scene will be defined by a scene script. Within a scene, dialogues and interactions between actors are defined, and lead to stage directions. If producing for an animated film, these directions are strictly mapped onto virtual actors by a skilled animator, who defines the way the virtual actors move and behave. When storytelling gets interactive, the user can influence the storytelling. In fact, in games as well as in constructivist scenarios for learning, users need to experience agency within a story. However, there are different levels at which to affect the outcome. In Fig. 3, the first author model (compare Fig. 2) has been extrapolated according to the need of introducing agency at each level. Opposed to the author, a participant is modelled who now may contribute to each level. The first implication for the author is that it’s not enough to just model a database of descriptions, but to add rules and models, which control an autonomous behaviour at each level in reaction to the participants. Then, it is possible to think of gradations of granted agency versus authored determination. Within Fig. 3 this is indicated by the sliders between control and autonomy at each level. The levels rather represent conceptual stages for authoring than elements of software architecture, though there are parallels to architectures of a number of existing systems of game and story engines. Semi-autonomy occurs on the edge between predefined factual information and rules for each level. The more rules on one level, the more agency can be experienced by potentially affecting the respective level. For example: It is imaginable that participants only experience agency on the lowest level, as a feeling of presence in a scenario. In this case, everything is predefined, but avatars would still react with nonverbal cues to the visitor and recognize her, comparable to a virtual cursor that shows a live status. At the conversation level, participants can for example have agency in an entertaining and informative chatbot dialogue with the characters. They may even not be able to affect anything in the story logic, but participate at dialogue level with speech acts. Agency at scene level would mean to have real choices about the outcome of a scene, for example, the story of the game would have to change according to user’s actions. On the top level, players would influence the whole story of the application, if the "agency slider" would be at a 100% to the right. For example, a simulation such as "The Sims" (Electronic Arts) can be put into the classification here. For factual knowledge transfer in a didactic lesson situation, the highest level could stay predefined, while the lower levels allow for conversational interaction, however constrained. If authors only provide a rule base with little pre-scripted structuring, they achieve a conceptual model more like an exploration or gaming experience depending completely on the action of the player. While arranging the bias at each level to various slider positions, several abstract genres of Interactive Digital Storytelling can be rebuilt in the model, which helps to specify exactly what kind of user experience an application shall provide. It is a conceptual model that can be used to classify story-related games, and it particularly supports authors coming from linear media, stepping into interactive storytelling. 3 Further Work In the full paper, I will also tackle related work while comparing with other theoretical models between games and stories, including references of the taxonomies of C. Lindley, M. Leblanc, J. Klabbers, B. Laurel, C. Pearce, and traditional classifications such as of R. Caillois. I will give more examples how existing products of Interactive Storytelling fit into the classification, and raise the question if new genres have to be defined particularly for Interactive Digital Storytelling. Literature Caillois, Roger: Man, Play and Games. (orig.: Les Jeux Et Les Hommes 1958) University of Illinois Press, Reprint (2001) Hunicke, Robin; LeBlanc, Marc; Zubek, Robert: MDA: A Formal Approach to Game Design and Game Research. In: Workshop Proceedings: Challenges in Game AI. 19th National Conference on Artificial Intelligence AAAI (2004) Klabbers, Jan H.G.: The gaming landscape: A taxonomy for classifying games and simulations. In Copier & Raessens (Eds.) Level up: Digital Games Research Conference. Utrecht University (2003) Lindley, Craig: Narrative, Game Play and Alternative Time Structures for Virtual Environments. In: Proc. TIDSE 2004, Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, Springer LNCS vol. 3105, 2004 Pearce, Celia: Emergent authorship: The next interactive revolution. In: Computers & Graphics 26, p. 21-29 (2002) Spierling, U.: Conceptual Models for Interactive Digital Storytelling in Knowledge Media Applications. In: Proc. TIDSE 2004, Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, Springer LNCS vol. 3105, 200

‘IMPLICIT CREATION’ – NON-PROGRAMMER CONCEPTUAL MODELS FOR AUTHORING IN INTERACTIVE DIGITAL STORYTELLING

Interactive Digital Storytelling (IDS) constitutes a research field that emerged from several areas of art, creation and computer science. It inquires technologies and possible artefacts that allow ‘highly-interactive’ experiences of digital worlds with compelling stories. However, the situation for story creators approaching ‘highly-interactive’ storytelling is complex. There is a gap between the available technology, which requires programming and prior knowledge in Artificial Intelligence, and established models of storytelling, which are too linear to have the potential to be highly interactive. This thesis reports on research that lays the ground for bridging this gap, leading to novel creation philosophies in future work. A design research process has been pursued, which centred on the suggestion of conceptual models, explaining a) process structures of interdisciplinary development, b) interactive story structures including the user of the interactive story system, and c) the positioning of human authors within semi-automated creative processes. By means of ‘implicit creation’, storytelling and modelling of simulated worlds are reconciled. The conceptual models are informed by exhaustive literature review in established neighbouring disciplines. These are a) creative principles in different storytelling domains, such as screenwriting, video game writing, role playing and improvisational theatre, b) narratological studies of story grammars and structures, and c) principles of designing interactive systems, in the areas of basic HCI design and models, discourse analysis in conversational systems, as well as game- and simulation design. In a case study of artefact building, the initial models have been put into practice, evaluated and extended. These artefacts are a) a conceived authoring tool (‘Scenejo’) for the creation of digital conversational stories, and b) the development of a serious game (‘The Killer Phrase Game’) as an application development. The study demonstrates how starting out from linear storytelling, iterative steps of ‘implicit creation’ can lead to more variability and interactivity in the designed interactive story. In the concrete case, the steps included abstraction of dialogues into conditional actions, and creating a dynamic world model of the conversation. This process and artefact can be used as a model illustrating non-programmer approaches to ‘implicit creation’ in a learning process. Research demonstrates that the field of Interactive Digital Storytelling still has to be further advanced until general creative principles can be fully established, which is a long-term endeavour, dependent upon environmental factors. It also requires further technological developments. The gap is not yet closed, but it can be better explained. The research results build groundwork for education of prospective authors. Concluding the thesis, IDS-specific creative principles have been proposed for evaluation in future work

Können Computerspiele Geschichten erzählen?

Prof. Dipl-Des. Ulrike Spierling studierte Kommunikations-Design in Darmstadt, und bearbeitete durch ihre Tätigkeit am Fraunhofer-Institut und Zentrum für Graphische Datenverarbeitung in Darmstadt auch zahlreiche Informatik-Forschungsprojekte im Bereich der interaktiven Unterhaltung und der Mensch-Computer-Interaktion. Seit 2002 ist sie Professorin für Mediendesign an der FH Erfurt. Ihr aktuelles Forschungsgebiet ist „Interactive Storytelling“. Spielerische Interaktionen können mehr sein als einzelne „Actions“ per Knopfdruck. Sie können der zwischenmenschlichen Kommunikation ähneln und komplexe Handlungen und Beziehungen zu Spielfiguren bewirken. Der Vortrag berichtet über aktuelle, interdisziplinäre Forschung im Bereich des interaktiven Geschichten-Erzählens. Um als Spieler in einer virtuellen Geschichte Einfluss nehmen zu können, muss sich die Geschichte automatisch anpassen können. Deshalb ähnelt das Schreiben einer solchen Geschichte zunehmend dem Programmieren von Handlungen. Im Vortrag werden die dadurch entstehenden Besonderheiten für die Konzeption geschildert sowie Forschungsbeispiele gezeigt

Social Life Cycle Assessments: A Review on Past Development, Advances and Methodological Challenges

Society’s interest in social impacts of products, services and organizational behaviors is rapidly growing. While life cycle assessments to evaluate environmental stressors have generally been well established in many industries, approaches to evaluate social impacts such as Social Life Cycle Assessment (S-LCA) lack methodological consistency and standardization. The aim of this paper is to identify past developments and methodological barriers of S-LCA and to summarize how the automotive industry contributed to the advancement or application of this method. Therefore, a qualitative content analysis of 111 studies published between 2015 and 2020 is used to gather information on past scientific and political milestones, methodological barriers impeding S-LCA and the participation of the automotive sector. The review shows that a broad range of sectors such as the automotive industry contributed to the testing and advancement of S-LCA in the past but that S-LCA remains a young and immature method. Large-scale application is impeded by major barriers such as the variety of impact categories and sub-categories, the lacking integration of the Sustainable Development Goals (SDGs), issues of linking LCA structures to social phenomena or the difficult tracking of social impact pathways. Further research on standardization possibilities, the connection to political social targets and the testing of methods is necessary to overcome current barriers and increase the applicability and interpretability results

Interactive Digital Storytelling: Towards a Hybrid Conceptual Approach

In this contribution, “Interactive Digital Storytelling ” is viewed as a hybrid form of game design and cinematic storytelling for the understanding and creation of future learning and entertainment applications. The result of several practical experiments in the design of interactive agents for storytelling, a formal design model is presented that provides a conceptual bridge between both traditional linear narrative techniques, as well as emergent conversations with virtual characters. This model describes several levels of apparent semi-autonomy in an agent-based artefact and agency experienced by participants, which draw- in the broadest sense-from Aristotle’s elements of structure in drama. In summary, thinking models for several design levels of interactive narrative are pictured from a creator’s point of view

Analyse der Anwendungsmöglichkeiten von Non-Photorealistic Rendering Techniken

Die Arbeit untersucht die Eigenschaften von sog. "Non-Photorealistic Rendering"-Techniken (NPR), und versucht neue Potentiale der Anwendungsmöglichkeiten dieser aufzuzeigen. NPR-Techniken sind bislang primär auf die Simulation "natürlicher" Medien, d.h. den klassischen Illustrations- und Maltechniken der bildenden Kunst, ausgerichtet. Ein Framework für die unterschiedlichen Techniken besteht jedoch nicht. Die Studienarbeit versucht, ein Framework für NPR-Techniken zu skizzieren. Hierzu werden neben einer Übersicht über die gängigsten Verfahren auch die Fundamente dieser Rendering-Techniken untersucht, wie etwa Kunstgeschichte und die Grundlagen visueller Kommunikation. Obschon die existierenden NPR-Verfahren durch ein breites Spektrum an Lösungswegen bestechen, so ist doch allen der Gedanke der grafischen Abstraktion gemein. Hierin sieht die Studienarbeit einen Ansatz für ein NPR-Framework, das die NPR-Techniken umfaßt und automatisiert. Hierzu wird versucht, Abstraktion formal zu definieren und "Meßmethoden" für den Abstraktionsgrad einer grafischen Darstellung herzuleiten. Die Arbeit schließt mit einem Systemmodell, das die Komponenten eines Frameworks beschreibt und ihre Arbeitsweise spezifiziert

'Implicit creation' : non-programmer conceptual models for authoring in interactive digital storytelling

Interactive Digital Storytelling (IDS) constitutes a research field that emerged from several areas of art, creation and computer science. It inquires technologies and possible artefacts that allow ‘highly-interactive’ experiences of digital worlds with compelling stories. However, the situation for story creators approaching ‘highly-interactive’ storytelling is complex. There is a gap between the available technology, which requires programming and prior knowledge in Artificial Intelligence, and established models of storytelling, which are too linear to have the potential to be highly interactive. This thesis reports on research that lays the ground for bridging this gap, leading to novel creation philosophies in future work. A design research process has been pursued, which centred on the suggestion of conceptual models, explaining a) process structures of interdisciplinary development, b) interactive story structures including the user of the interactive story system, and c) the positioning of human authors within semi-automated creative processes. By means of ‘implicit creation’, storytelling and modelling of simulated worlds are reconciled. The conceptual models are informed by exhaustive literature review in established neighbouring disciplines. These are a) creative principles in different storytelling domains, such as screenwriting, video game writing, role playing and improvisational theatre, b) narratological studies of story grammars and structures, and c) principles of designing interactive systems, in the areas of basic HCI design and models, discourse analysis in conversational systems, as well as game- and simulation design. In a case study of artefact building, the initial models have been put into practice, evaluated and extended. These artefacts are a) a conceived authoring tool (‘Scenejo’) for the creation of digital conversational stories, and b) the development of a serious game (‘The Killer Phrase Game’) as an application development. The study demonstrates how starting out from linear storytelling, iterative steps of ‘implicit creation’ can lead to more variability and interactivity in the designed interactive story. In the concrete case, the steps included abstraction of dialogues into conditional actions, and creating a dynamic world model of the conversation. This process and artefact can be used as a model illustrating non-programmer approaches to ‘implicit creation’ in a learning process. Research demonstrates that the field of Interactive Digital Storytelling still has to be further advanced until general creative principles can be fully established, which is a long-term endeavour, dependent upon environmental factors. It also requires further technological developments. The gap is not yet closed, but it can be better explained. The research results build groundwork for education of prospective authors. Concluding the thesis, IDS-specific creative principles have been proposed for evaluation in future work.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Novel User Interface Technologies and Conversational User Interfaces for Information Appliances

The metaphor of a 'desktop' as a user interface was useful for the migration from mainframe applications to personal computers. But it must be re-evaluated when thinking about new concepts and implementations of user interfaces for modern working and home environments, especially mobile applications. The departure from tools that enable the user to directly manipulate content towards intelligent agent-based applications suggests a shift of paradigms. We propose a new generation of user interfaces extending the widely accepted WIMP metaphor by anthropomorphous user interface agents, conversational user interface elements, the inclusion of mobile information appliances, and appropriate multimodal interaction techniques. Initial prototype systems following this line have been successfully developed. In strategic projects with alliances of research and industry, these technologies will be further advanced into first products in the areas of mobile multimedia offices and electronic multimed..